UNBOUNDED VISCOSITY SOLUTIONS OF HYBRID CONTROL 

SYSTEMS 
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Abstract. We study a hybrid control system in which both discrete and continuous controls are 
involved. The discrete controls act on the system at a given set interface. The state of the system is 
changed discontinuously when the trajectory hits predefined sets, namely, an autonomous jump set 
A or a controlled jump set C where controller can choose to jump or not. At each jump, trajectory 
can move to a different Euclidean space. We allow the cost functionals to be unbounded with certain 
growth and hence the corresponding value function can be unbounded. We characterize the value 
function as the unique viscosity solution of the associated quasivariational inequality in a suitable 
function class. We also consider the evolutionary, finite horizon hybrid control problem with similar 
model and prove that the value function is the unique viscosity solution in the continuous function 
class while allowing cost functionals as well as the dynamics to be unbounded. 
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1. Introduction. The hybrid control systems are those involving continuous 
and discrete dynamics and continuous and discrete controls. Typical examples of 
such systems are constrained robotic systems [T] and automated highway systems 
[12] . For some more examples of such systems, see [B],[7] and references therein. 

In [6] , Branicky, Borkar and Mitter have presented a model for a general hybrid 
control system in which continuous controls are present and in addition discrete con- 
trols act at a given set interface, which corresponds to the logical decision making 
process as in the above examples. The state of the system is changed discontinuously 
when the trajectory hits these predefined sets, namely, an autonomous jump set A or 
a controlled jump set C where controller can choose to jump or not. They prove right 
continuity of the value function corresponding to this hybrid control problem. Using 
dynamic programming principle they arrive at the partial differential equation satis- 
fied by the value function which turns out to be quasivariational inequality, referred 
hereafter as QVI. 

In an earlier work [9], the authors have studied this problem and proved the 
local Holder continuity of the value function and have characterized it as the unique 
viscosity solution of the QVI. There the cost functionals were assumed to be bounded 
and hence the value function was also bounded. This was essential in the uniqueness 
proof, as the auxiliary function had to be modified a finite number of times to get 
the comparison result. In this paper we allow the cost functions to be unbounded. 
We use a different method to compare unbounded value functions, namely a suitable 
change of variable to reduce the unbounded function to a bounded one (See [3], for 
example ). We also tackle the uniqueness for the time dependent problem, using a 
test-function similar to the one introduced in [1]. 
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The paper is organized as follows: In section 2, we introduce the notations, as- 
sumptions and quasivariational inequality (QVI) satisfied by the value function. In 
section 3, we show that the value function is continuous. The next section deals with 
uniqueness of the solution of QVI. We give a comparison principle proof characteriz- 
ing the value function as unique viscosity solution of QVI , for the stationary case in 
section 4 and then for the time dependent case in section 5. 

2. Preliminaries . In a hybrid control system, as in [5], the state vector during 
continuous evolution, is given by the solution of the following problem 

(2.1) X(t) = fi(X(t),u(t)) 

(2.2) X(0) = x 

where x <E Q$ , a connected subset of M di , di 6 Z + for i 6 Z\ C Z + . Here fi : QixU — > 
Qi and the continuous control set is : 

hi = {u : [ 0, oo) — ► U | u measurable, U compact metric space. } 

The trajectory undergoes discrete jump when it hits predefined sets Ai the au- 
tonomous jump set and d, the controlled jump set, both subsets of M di . The 
trajectory starting from x £ Qi, on hitting Ai, can jump to a predefined destina- 
tion set Dj in another Euclidean space Qj and continue the evolution there. This 
jump is given by prescribed transition map g : Ai x V — > (J Di, where V is the discrete 

i 

control set. On hitting d the controller can choose either to jump or not to jump. 
If the controller chooses to jump, then the trajectory is moved to a new point in Dk, 
possibly in another space Ofc . This gives rise to a sequence of hitting times of (J Aj , 

3 

which we denote by Oi and sequence of hitting times of (J Cj , where the controller 

j 

chooses to jump which is denoted by 
We introduce the state space 

n := (Jf^ x {i}, ieZ 1 cZ + , 

i 

and the dynamics / : f2 X li — > CI. Actually f = fi with the understanding, X(t) — 
fi(X(t),u(f)) whenever x G fti. The predefined sets are 

A = U^ x {*} M C Oi C ]R d ' ; 
C = U d x {i} d CS1.C R d > ■ 

i 

D = |J Di x {i} Di QfliC lR di . 

i 

The trajectory of this problem, is composed of continuous evolution given by 
(|2.1|) between two hitting times and discrete jumps at the hitting times. We denote 
X(a~[ , by Xi, the point before an autonomous jump and g(X(a~[), v) by x[, after 
the jump. The controlled jump destination of X(£~ , u{-)) is X{&) , or simply, X(£j)'. 
For example, the dynamics for Of < ^ < is given by 

X(t) = f(X(t),u(t)) ai<t<^ k , 
X(a i )=X(at)=9(X(*r), v )= x ' i , 
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and then 



X(t) = f(X(t),u(t)) Z k <t<a i+1 
X(£ k ) = X(£) = X(C)', 



We give the inductive limit topology on £1 namely, 



(x n , i 



n 



) e £1 converges to (a;, i 



) S n if for some N large and V n > N 



i n =i x, x n € Oj, Oj C M d %ioi some i, and ||a;„ — 



With the understanding of above topology we suppress the second variable i from fl. 
We follow the same for A, C and D. We also denote by \x\ the quantity which is equal 
to H^Hjjdi if x e fli- 

We make the following basic assumptions on the sets A, C, D and on functions 
/ and g: 

(Al): Each Oj is closure of a connected, open subset of M di . 

(A2): Ai, d, Di are closed, and for all i and for all x € Di, \x\ < R. dAi,dCi are C 2 
and dA t D dVt, V i. 

(A3): g : A x V — > D is bounded, uniformly Lipschitz continuous map, with Lipschitz 
constant G with the understanding that g — {gi} and gi : Ai x V — > UjDj. 
(A4): Vector field / is Lipschitz continuous with Lipschitz constant L in the state 
variable x and uniformly continuous in control variable u. Also, 

(2.3) \f(x,u)\<F Vxefland Vugf/. 

(A5): Each cM^ is compact for all i, and for some ^ > 0, following transversality 
condition holds: 



where ((xq) is the unit outward normal to dAi at x . We assume similar transversality 
condition on <9Cj. 
(A6): We assume that 



where d is the appropriate Euclidean distance. Note that, above rules out infinitely 
many jumps in finite time. 

(A7): The control sets U and V are assumed to be compact metric spaces. 

Now 9 := (u(-),v,£i,X(£i)) is the control and total discounted cost is given by 



(2.4) 



/(x ,u)-CM <-2£o V x e dAi V we U 



(2.5) 



inf d(Ai, Ci) > (3 and inf d(A u A) > > 



I 2 



DC 




OC- 



-Acri 



f) 



i=0 



(2.6) 
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where A is the discount factor, K : £1 x U -^M + is the running cost, C a : A x V — > 
_ZR + is the autonomous jump cost and C c : C x I? — > _ZR + is the controlled jump cost. 
The value function V is then defined as: 

(2.7) V{x)= inf J(:r,u(-)> 

9e(UxVx[0,oo)xD) 

We assume the following conditions on the cost functionals: 
(CI): _ftT is nonncgative, continuous in the x variable with at most polynomial growth 
of degree k , with k and the growth being independent of i. K is uniformly continuous 
in u variable. 

(C2): C a (x,v) and C c (x,x') are continuous in both variables, uniformly continuous 
in x, uniformly with respect to v and x' respectively, and bounded below by C > 0. 
Moreover C a and C c have at most polynomial growth of degree k in the first variable, 
with k and the growth being independent of i. 

Note that under (CI) and (C2), value function is always non-negative and hence 
bounded below by 0. Using dynamic programming principle, one can show that the 
value function satisfies the following QVI in viscosity sense: 

Theorem 2.1. /Quasivariational Inequality/ Under the assumptions (Al — A7), 
(C1),(C2) and if A > kL, the value function V described in \2. 7| ) is continuous 
and has at most a polynomial growth of degree k. Moreover it satisfies the following 
quasivariational inequality in the viscosity sense: 

(MV(x) Viei 
mm{NV(x),-H(x,DV(x))} VieC (QVI) 
-H{x,DV(x)) V x en\A\JC 

For </>, a function defined on f2 and bounded below, M, N and H are given by 

M(f>(x) = inf {<f>{g(x, v)) + C a (x, «)}, 
N<j){x) = inf {<f>(x') + C c (x, x')}, 



H(x,p) = sup 

uEU 



-K(x,u) - f(x,u)-p 
A 



The condition A > kL in Thcorcm l2.1l is used (and needed in the general case) to 
ensure that the value function is locally bounded and has a polynomial growth. For 
the uniqueness result for the quasivariational inequality (cf. Section|4]), such condition 
is not playing any role. 

Now we consider the hybrid control problem, with autonomous and controlled 
jumps as before, but in finite horizon, namely for t £ [0, T\. In this case the dynamics 
depends on the time variable t and we also allow cost functionals to depend on t. We 
relax the global Lipschitzness assumption on / and assume only local Lipschitzncss. 
Further relaxing boundedness of /, we allow it to grow linearly. 

In order to simplify we assume that the cost C a , C c are as above independent of 
time. It is not difficult to check that, besides additional technical details, the case 
when C a , C c depend on t can be treated in a similar way, with suitable adaptations. 
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The total cost in the time dependent case is given by, 

T 

J(s,x,u(-),v,^,X^ i y)= f K(t,X(t),u(t))dt + ]T C a {X(*i),v) 
(2-8) + C c (X(^),X(^y) + h(X x (T)) 

s<£i<T 

where K : [0, T] xilxU -^M+ is the running cost, C a ■ AxV — ► M+ is the autonomous 
jump cost, C c : C x D — > _ZR + is the controlled jump cost and h is the terminal cost. 
The value function V is then defined as: 



(2.9) 



V(s,x)= inf J(s,x,u(-),v,€i,X(€i)') 

0e(UxVx[s,T)xD) 



The precise assumptions are as follows: as before we assume (A1)-(A3), (A6) and 
(A7). Because / and K are time dependent we further assume the following: 
(A4)': Vector field f(t,x,u) is locally Lipschitz continuous in the state variable x 
and uniformly continuous in the rest of the variables. Its linear growth is given by 

|/(f, x, it) | < F(l + \x\) V x e ft and V (t, u) e [0, T] x U. 

(A5)': Each dAi is compact for all i, and for some > 0, following transversality 
condition holds: 



(2.10) 



f{t,x ,u) ■ C(x Q ) < -2£„ V x G dA t V (t,u) e [0,T] x U 



where C( x o) is the unit outward normal to dAi at x . We assume similar transversality 
condition on dCi. The cost functionals are assumed to satisfy 

(CI)': K is nonnegative and continuous in all the three variables; moreover K is 
uniformly bounded for bounded \x\. 

One can derive the DPP for time dependent problem which is given as follows: 

For s' such that s < s' < T, 



V(s,x) = inf 



s'<(cti/ 



A£l) I J K(t, X(t),u(t))dt + V(s', X(s')) 
+hi<(s'A, 1 ) I J K(t,X(t),u(t))dt + V(Z u X(tr)) 



It can be verified that the quasivariational inequality (QVI-T) satisfied by V(s, x), 
is given by 



V-MV = in(0,T)x,4 
max{V - NV,-V s + H(s,x,DV)} = in (0,T) x C 

-V s + H(s,x,DV) = in (0,T) x fi\ AuC 



(QVI - T) 



G 



where for <fi, a function denned on ([0, T] x SI) and bounded below, M, N and H are 
given by 

M<f>(t,x) = inf {<f>(t,g(x,v)) + C a (x,v)} V x G A 
N<j>(t,x)= m£ {4>(t,x') + CJx,x')} Vi eC 

x f GD 

H(t, x,p) — sup{— K(t, x, u) — f(t, x, u) ■ p} 

ueu 

For the terminal condition, it turns out to be more complicated than it is usually 
the case because of the possible (or imposed) jumps. In fact, the terminal data is 
obtained by solving the stationary problem 

V{T, x) - MV{T, x) = V x e A ~\ 
(2.11) max{V(7» - NV(T,x),V(T,x) - h(x)} — VigC > 

V(T,x)-h(x) = Vxen\AuC. J 

In order to have a continuous solution V (and therefore a continuous terminal V(T, x)), 
we have to assume 

(Dl): Equation (|2.11[) has a (unique) bounded from below solution h which is uni- 
formly continuous for bounded 

We point out that the key property in this assumption is the existence of such 
uniformly continuous solution (uniformity means really uniformity in i) while unique- 
ness is rather easy to obtain. This is really an assumption on h. To go further in this 
direction, we also remark that if C c satisfies 

C c {x, y) < C c (x, z) + C c (z, y) for any x G C,y G D and z £ C n D . 

which means that to jump once is always better that to jump twice, then h can be 
built in the following way 

(i) h is given except on A and C. 

(ii) For points in C , set h = min(h, Nh). 

The above condition implies that, actually, Nh < N(Nh) onCflD and therefore the 
right inequality holds in (|2.11[) 

(iii) For points in A, set h = Mh. 

This is well-defined since h is known on D. 

Assumption (Dl) consists in assuming that this function h is uniformly continuous 
for bounded which is both a continuity assumption on h and a compatibility 
condition on its value near OA and dC. If for example, CnD is empty, then assuming 
h = Mh on dA and h < Nh on dC in addition to the uniform continuity of h on 
bounded subsets of fi, will ensure that of h, since h = h. One can construct easy 
examples in one dimension to show that if C PI D is nonempty, the situation gets 
complicated and the conditions on h are not that transparent. 

Once h is known, we are left with a more classical terminal condition 

V(T, x) - h(x) = VzGQ. 

3. Continuity of the value function. Let the trajectory given by solution 
of (|2.1|) and starting from the point x be denoted by X x (t,u(-)). Since x G CI, in 
particular it belongs to some Jl^. Then we recall from theory of ordinary differential 
equations: 

(3.1) \X x (t,u(-)) - X z (t,u(-))\ <e Lt \x-zl 

(3.2) \X x (t,u(-)) - X x (t,u(-)) < F\t -i\,\X x (t,u(-))\ < \ x \e Lt +j(e Lt -l) 
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For the derivation of these, see for example, ([5] , Appendix, Chapter 5). Here F 
and L are as in (A4) . Now we can proceed as in [5] using the transversality condition 
(A5) and conclude the continuity of the value function. Moreover, as running cost 
K, and discrete jump costs C a and C c have polynomial growth of order k, one can 
see that total cost J also has at most the same growth because of the assumption 
kL < A. Then it follows that the value function V also has a polynomial growth of 
order k, uniformly in all Qi. 

For i] > 0, we introduce the following spaces of functions : we denote by E r ,(Cl) 
the space of semicontinuous functions u defined on f2 and bounded below and such 
that, in each 17, 

lim u{x)e~^ = 0, 

\x\ — >oo 

the limit being uniform w.r.t. i. Then E(Q) is defined by 

E(n) = {j, 1<X / F e 71 . 

In section 4, we prove the uniqueness of viscosity solution of QVI in this function 
class E to which the value function belongs. 

In the case of finite horizon problems, the continuity of the value function is a 
more delicate problem : indeed, in the infinite horizon case, the fact that time is 
unbounded allows to use the transversality condition (A5) and to compensate the 
jumps, perhaps with a delay, and a priori this is not possible anymore in the finite 
horizon case. This shows that the problem is even more important when t is close 
to the terminal time T and this explains the above discussion about (Dl) and the 
terminal condition h. 

To prove directly (and without to many technicalities) that the value function 
is continuous requires (Dl) but also the uniform continuity on bounded subsets of 
fi of the functions K, C a , C c , which are rather strong assumptions. An alternative 
proof (but essentially with the same assumptions) is to show that the upper and lower 
semi-continuous envelopes of V are respectively viscosity sub and supersolutions of 
(QVI-T) and to use the comparizon result of Section 5; this provides a slightly simpler 
proof. 

4. Uniqueness - Infinite Horizon Problem. In this section, we state and 
prove a comparison between sub and supersolutions of QVI, which belong to E(Tt). 

Theorem 4.1. Assume (Al) - (A7) and (CI) - (C2). Let u 1: u 2 € E(ft), be 
respectively upper semicontinuous subsolution and lower semicontinuous supersolution 
of the quasivariational inequality given by (QVI) in the viscosity sense. Then U\ < u 2 
in f2. 

Proof. We complete the proof in 4 steps. In step 1, we convert the unbounded 
value functions ui, u 2 into bounded functions W\, w 2 by a suitable change of variable. 
We derive the modified quasivariational inequality satisfied by w\ and w 2 . In the 
following 2 steps, for fixed fi S (0, 1), close to 1, we argue on /iwi and w 2 : in Step 
2, we examine the possibility of having approximate suprema at points of A U C and 
the actual consequences of such facts, while, in Step 3, we use the previous results 
to reach a contradiction. In Step 4, we conclude the comparison of u\ and u 2 using 
steps 2 and 3. 

Step 1: Let u\,u 2 be respectively upper and lower semicontinuous sub and super- 
solutions of the QVI in E{£L). Fix < r/ < -p such that both ui,u 2 lie in E v , where, 
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as above, A is the discount factor and F is the bound on dynamics /. Define 

tui(a;) = u 1 {x)e~^ x \ w 2 (x) = u 2 (x)e^ x \ 

where £ : — > R + is a smooth function such that, \D£\ < 1 for all x G O, £(ar) = 
if \x\ < R and £(x) = (1 + |x| 2 ) 1/2 if \x\ > 2R where i? is given by (A2). Since £ 
behaves like |x| at infinity and u\,u 2 G -E^, w\(x), W2{x) are bounded and converge 
to when x tends to infinity in each fij, uniformly w.r.t. i. 

The functions W\,W2 are respectively upper and lower semicontinuous sub and 
supersolution of a QVI which can be first written as 



w{x)e 



' M{w{x)e r >^) VieA 
min {N(w(x)ertW), -H(x, D(w(x)e^^))} V x E C 
-H(x,D(w(x)ertW)) Vxen\A\JC 



But, recalling the facts that |x| < R if x G D and that g maps Ax V into D, we can 
simplify this QVI in the following way 



w(x) 



Mw(x) V x e A 

min^Nw(x),-H(x,w(x),Dw(x))^ V x G C 
-H{x,w{x),Dw{x)) Vxen\AuC 



where 



Mw(x) = e-rtW mt{w(g{x,v))+C a (x,v)} 



vEV 



Nw(x) = e-rtW inf {w(x') + C c (x, x')} 
x'eD 



H{x,w,Dw{x)) = \ (sup{-/(x,u) • (Dw(x) +r 1 w(x)D((x))} - e-^K(x,u)}\ 

Indeed, by definition of £ and for i = 1,2, Wi(g(x, v j) = Ui(g(x, v)) for any x E A and 
»eV, and, in the same way, Wi(x') — Ui(x') for any x' G D. 

Using the definition of i7 and the assumptions on the cost functionals and dy- 
namics we can show that 

\H(y,w 2 {y),P2) - H(x, Wl (x), Pl )\ < (L/X)\x - y\\ Pl \ + {F/X)\ Pl (x) - p 2 (y)\ 
(4.1) + (r]F/X)\wi (x) - w 2 (y)\ + (1/X)lj r (\x - y\) 

+ (v/X)\w 2 \ snp{\DZ(x) ■ f(x,u) - D£{y) ■ f(y,u)\} 
ueu 

where K(x, u) = e~ 71 ^ x ^K(x, u) and uj^ is the modulus of continuity of K. 

Now we fix /z G (0, 1), close to 1, and prove the comparison result for fiwi and 
w 2 . To do so, we assume, by contradiction, that supq(/zwi(x) — w 2 (x)) > or, in 
other words 

supsup(/xwi(x) — w 2 (x)) = m > 0. 

j Qj 

We first recall that wi(x), w 2 (x) converge to when x tends to infinity in all flj, 
uniformly w.r.t. j; therefore each sup n . is indeed achieved at some point. But since 
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the set of indices j is infinite, we have anyway to argue with approximate supremums 
(of course, if this supremum is attained at some finite point then the following proof 
gets simplified). We consider k > small enough (a more precise estimate on its size 
will be given later on) and let i and x K be such that 

(4.2) sup(iiwi(x) - w 2 (x)) = nwi(x K ) - w 2 (x K ) > m - k > — - > 0. 

Note that x K remains bounded : indeed we can find R m such that 

|/Ltwi(ir)| < m/10 and 1 102(2;)! < m/10 for |z| > R m for all x g flj and for all j. 

Then, x K being the maximum of fiwi — w 2 in and m/2 < m — k < (fJ/Wi — w 2 )(x K ) 
will imply that \xk\ < R m - 

Next we are going to consider three different cases, namely, 

1. x K €E Ai, 

2. x K e Ci and w 2 (x K ) > Nw 2 (x K ) 

3. Either x K € fli \ (Ai U Ci) or x K G Ci and w 2 (x K ) < Nw 2 (x K ). 
We consider cases 1 and 2 in the next step and case 3 in Step 3. 

Step 2: We first consider the case 1 namely x K E Ai. For w%, w 2 , viscosity sub and 
supersolutions, the conditions on dA are satisfied only in the viscosity sense: For the 
subsolution 

(4.3) min{u'i + H (x, Dwi,wi), wi — Mw{\ < on dA 
For the supersolution, 

(4.4) max{iu 2 + H (x, Dw 2 , w 2 ), w 2 — Mw 2 } > on dA. 
Hence we need to rule out the bad inequalities for w\,w 2 , namely 

Wi > Mwi or w 2 < Mw 2 

on dA. Our assumption (A5) helps us to avoid the above situation and to conclude 
that the right inequalities for wi,w 2 hold on dA. To do so, we need the following 

Lemma 4.2. If w\ and w 2 are viscosity sub and supersolutions of QVI and w\, w 2 
are u.s.c and l.s.c on Q respectively, then the functions Mw±,Nwi and Mw 2 , Nw 2 
are respectively upper and lower semi-continuous. Moreover, for any x € dA, we have 

wi(x) < Mu>i(x), w 2 (x) > Mw 2 (x) 

and for any x € dC, we have wi(x) < Nwi(x). 

We postpone the proof of this lemma to the end of the section. 
We now conclude the proof by using Lemma l4~2l If x K e A then, using Lemma l4~2l 
for boundary points, we have 

wi(x K ) < Mwi(x K ) and w 2 (x K ) > Mw 2 (x K ) 

By definition of M, and compactness of control set V, we can find vq such that, 

Mw 2 (x K ) = e~ r >^ x ^{w 2 (g(x K ,vo)) + C a (x K ,v )} 
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For all controls v £ V and hence in particular for vq we have, 

Mwi{x K ) < e-i^iwiig^vo)) + C a (x K ,v )} 

Hence, 

fj,Wi(x K ) - w 2 (x K ) < e~ r, ^ XK '{(j,w 1 (g(x K ,Vo)) - w 2 (g(x K ,v )) - (1 - fi)C a (x K ,v )} 

< e-^ x ^{nwi{g(x K ,Vo)) ~ w 2 (g(x K , v )) - (1 - M )C"} 

< e -"^{m- (1-m)C"} 

This inequality and (|4. 2[) imply 

m- K < (J,Wi(x K ) - w 2 (x K ) < e'^ x ^{m - (1 - fi)C'} < m - (1 - /i)C 
This is a contradiction if we choose k such that 



(4.5) (1 - y)C? > ft- 

Thus, for such a choice of ft, x K cannot be on Aj. 

Next we consider the case when x K e d. If W2(i K ) > Nw 2 (x K ), we use Lemma [4.2l 
to deduce w\(x K ) < Nw\(x K ) and a similar argument as above leads to a contradiction 
under the same condition on k. Therefore we may assume without loss of generality 
that w 2 (x K ) < Nw 2 (x K ) which means that we are in the third case. 
Step 3: We examine here the third (and last possible) case. To do so, we introduce 
the function $ : fij x f2j — > M by 

<&{x,y) = nwx(x) - w 2 (y) - -\x - y\ 2 - \x - x K \ 2 

e 

where e is a small positive parameter, devoted to tend to 0. Since <I> goes to — oo as 
|x| goes to oo, there exists a maximum point (xo,yo) of <£. Note that, 

(4.6) — < (iWi(x K ) - w 2 (x K ) = $(x K ,x K ) < sup$(a;,y) = $(x ,y ) . 

The following estimates, listed in a lemma, are standard in the theory of viscosity 
solutions and can be derived from the definition of and (xo,yo). For example see 
[7], Chapter 3, Theorem 2.12. Only the last property stated in the lemma, is non- 
standard. 

Lemma 4.3. 

(i) \xq — j/o I < y/Ce for some constant C depending only on the L°° bounds on 
wi and w 2 . 

(it) 1 ■ ► as e — ► 

e 

(Hi) x ,y -> x K , W\(xq) -> Wi(x K ), w 2 (y ) -> w 2 (xk) when e^O, 

(iv) If x K <G Ci and w 2 (x K ) < Nw 2 (x K ), w 2 (yo) < Nw 2 (yo) if e is small enough. 

We leave the proof of the point (iv) of this lemma to the reader: it just uses the 
fact that w 2 (yo) — > w 2 (x K ) when e — > together with the lower semi-continuity of 
Nw 2 coming from Lemma 14.21 

Now define the test functions <j)\ and (j> 2 by 

<t>i{x) = w 2 {y Q ) + -\x-y \ 2 + \x-x K \ 2 and fa(y) = fiWi(x )--\x -y\ 2 ~\x -x K \ 2 
e e 
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The function [j,Wi — <p\ attains its maximum at Xq and, for e small enough, Zo € A^ 
because x — > x K S f2{ \ ; therefore QVI implies 

/zwi(x ) + sup Y{~/(^o,u)-(A i ^^i(^o) + W^i(2 ; o)£ , C(2 ; o))-Aie _ ' ,c(;Eo) i ; i:(a;o,'a)} < . 
wee/ A 

On the other hand, w 2 — <{) 2 attains its minimum at yo € Qi\Ai and w 2 (yo) < Nw 2 (yo) 
if yo € therefore, by using definition of viscosity supcrsolution, we have 

Mvo) + t sup{-/(j, , U ) • (.002(2/0) + VW 2 (y )D^y )) - K(y ,u)} > . 

Denoting byp e = - - — and substituting Dcj)i(xo) = p e + 2(xq — x K ), D(f> 2 (yo) = 
p e in the above and estimating as in (|4.1[) , we get, 

(A - j]F)(pw 1 (x ) - w 2 (y )) < L\%o - Vo\{\Pe\ + v\w 2 \(F + L)\x - y | 

+ (1 - /j^uj^ {\x - y \) + 2F\x - x K \ . 

Now remembering A — r/F > 0, 

(A - nF)§(x ,yo) = (A - r)F){fiw 1 (x ) - w 2 (y ) - ^\x ~ y \ 2 } 

< (A - t]F){^wx{xq) - w 2 (y )) 

< L\x - yo\{\ 2{X °~ yo) } + V(F + L)\w 2 \\x - y \ + 
(1 - li/ug^xo - yo\) + 2F\x - x K \ 

Using the lemma, we can choose e such that RHS of the above inequality can be made 
arbitrarily small, which will contradict (|4.6[) namely, 

in 

$(zo,yo) > y > 0. 

Thus, in each of the three steps, we arrive at a contradiction to (|4.2[) and we 
conclude that, 

supsup(^iWi(x) — w 2 {x)) < 0. 

Now sending /j to 1 we get the required comparison between w\ and w 2 , namely, 

w\(x) < w 2 {x) for all x in SI. 

Step 4: From Step 3, we have concluded that 

W\(x) — w 2 (x) < V x 6 £1 

Now, by definition of u>i , w 2 , 

Wl {x) = Ul {x)e^ x \ w 2 {x) = u 2 (x)er^ x \ 

hence, 

Ul (x) - u 2 (x) = e^{wi{x) - w 2 (x)} < V xett 
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Thus, we have the comparison between u\ and 112, which are solutions of the original 
quasivariational inequality. 

In order to complete the proof, we have to provide the 

Proof of Lemma 14. 21 This uses the idea in lemma 5.2 (page 113) of [3|. Suppose 
that for some x £ dAi 

w\(x) > Mw\{x). 

Let the distance function from Aj be denoted by d(-). Consider the test function, 

\x — y\ 

<t>{y) = w i{y) Cd(y) 

for positive parameters e and C, to be fixed suitably later on. Let y £ be the max of 
cf> in f2j. Then in particular, 

wi(y £ ) > wx(x) 

for each e and y e — > x. Hence 

liminf Wx(y e ) > wi(x). 
Using the uppersemicontinuous property of w\ , 

lim sup wi (y £ ) < w\(x). 

Thus 

limwi(y £ ) = wi(x) 
By our assumption w±(x) > Mw\{x). Since Mw is u.s.c, we have 

wiiVe) > Mw\{y e ) 
for e sufficiently small. If y £ S dAi, then we have by (|4.3II . 

Wl (y £ )+supl{-(/(y £ , u ), 2(x ~ y£) -Cn( 2 / £ )+7 ?Wl ( ?/£ )^( ? / £ ))- e -"^)X( ? / £ , M )} < 
u A e 

where for the distance function d(-) from A, Dd{x) = n{x). Now, we divide by C and 
let e — > and C — > 00, in such a manner that C-^e — ► 00. Recalling that ^^| e ^ — > 0, 
we get 

sup{(/(x,u),n(a;))} < 0, ^ (/(x, u), n(x)) < V a£W. 

u 

As n(x) = — (,{x), appearing in (A5) this conclusion contradicts {A5). Hence our 
assumption cannot hold. 

In a similar manner arguing with the test function. 

4>(y) = My) + + «%) 

we get the required inequality of 11)2. Thus the proof is complete. □ 

This theorem characterizes the value function of hybrid control problem as the 

unique viscosity solution of the QVI in the function class E defined earlier. We would 

like to remark the following. 

Remark 4.4. We can extend the uniqueness result to the hybrid game theory 

problem with unbounded value functions. The case of bounded value functions was 

treated in UOf - 
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5. Uniqueness - Finite Horizon Problem. The comparison proof for finite 
horizon problem is proved by building test-functions, which go to infinity at the 
boundary of the domain of comparison and tend to zero in a specific subset of the 
interior. These are similar to the ones in Ley [11) and Barles, Biton and Ley[4] 
where they are introduced as "friendly giants" , either for proving "finite speed of 
propagation type results" for first-order equations or for proving that the comparison 
holds for second-order equations. Here it is extended to quasi-variational inequalities 
involving HJB equations. 

First we announce the uniqueness theorem. 

Theorem 5.1. - Assume (A1-A3), (A4)', (A5)', (A6), (A7) and (CI)', (C2). Let 
Ui,u 2 be respectively a u.s.c. subsolution bounded from below and a l.s.c supersolution 
bounded from below of the quasivariational inequality (QVI-T) in (0,T) x O. 7/ii 1 ,u 2 
satisfy: for any r > 0, u%, u 2 are bounded on [0, T] x {x S 17 ; |x| < r} and 

(5.1) limsup sup [ux(t, x) — x)] < , 

t^T \x\<r 

then u\ < U2 in (0,T) x 57. 

We point out that (|5.ip is a non trivial assumption since we work in Q = (J 57^ x {i} 

i 

because of the dependence in i; roughly speaking, it means that the terminal data is 
assumed with a certain uniformity w.r.t i. This assumption is satisfied by the value 
function of the control problem under mild additional assumptions on the data as we 
show it at the end of the section. 

Proof. Since u\,U2 are both bounded from below, we can assume as well that 
they are positive since we can add the same positive constant to both of them and 
the resulting functions still solve (QVI-T) in (0, T) x 17 (of course, (|5.ip also holds.) 

We want to prove that for fixed fi close to 1 in (0, 1), 

w = fxui — U2 < in [0, T] x 57 . 

For that, we divide the interval [0, T] into finite number of sub-intervals of length less 
than a certain constant and prove the comparison on this subinterval first and then 
extend the result to [0, T] by repeating the argument on each of the subintervals. 

Let us set K := F(l + R + (3), where F is as in (A4)', R is the bound for D, 
introduced in (A2) and f3 is as in (A6). We choose T such that 

(5.2) T - To<min ^4 } 

The first step consists in proving that w < on [To,T] x D. To do so, we introduce 
a neighbourhood of D, namely 

D := {x e fl : d(x,D) < (3 } 

for (3 > as before and we set 

m(To) := sup w(t,x). 

[T ,T]xD 

Because of assumption (|5.1j) and since 



Ul(t,x) = [n — l)ui(t,x) + (Ux - U2){t,x) < (Ux ~ U2)(t,X), 
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there exists r, T < r < T such that 

(5.3) m(r) < 

where C" is the lower bound for the cost functionals introduced in (C2). We will show 
that w is a subsolution of the variational inequality 

(5.4) min{-w t (t, x) - F(l + M)|£>H , w(t, x)} = 

in D T := (r, T) x I? and from that we will conclude w < on [r, T] x Z? by using 
"finite speed of propagation type properties" (cf. for example Ley [11]). Once this is 
true for every such r, it will follow that w < in [T , T] x D. 

From this first result, we will deduce, in a second step, that w < on [T , T] x A. 
Finally the third step will be devoted to proving first that id is a subsolution of (|5.4[) 
in [T , T] x n \ A and then w < in [T , T] x Q. 

Proof of Step 1 : Let $ S C 2 ((0, oo) x f2 K ) and let (i, x) & D T be a, strict local 
maximum point of w — $ = /xui — M2 — $ in Z) r . We want to prove that 

min{-$t(i,5) - F(l + x)\ , < . 

If w(i, x) < 0, there is nothing to prove. 

If uu(i, x) > 0, since C H D need not be empty, we first remark that 

(5.5) u 2 (i, x) < Nu 2 (t,x) if x E C K 

Indeed, otherwise, since u\(t, x) < Nui(i,x), we would have, for some x' G D 

{pu\ - u 2 )(i, x) < (fiui - u 2 )(i, x') + ((j, — l)C c (x, x') < m(r) + (fx — 1)C", 

which contradicts the fact that w(t, x) > 0, because of our choice of r, satisfying (|5.3|) . 
For e > 0, we consider for some p such that (i — p,i + p) x B(x, p) C D T , 

\x — 

max {pui(t, x) - u 2 (t,y) - &(t,x) t, — } • 

(t~p,t+p)xB(x, P ) 2 e z 

It can be shown that the above maximum is achieved at points (x £ ,y £ ,t £ ) such that 

— I ^ s — Ve I 

(t s , x e , y e ) — * (t, x, x) and — - — ^— > as e — > 0, 

u 2 {t E ,y E ) -> u 2 (f,S). 

Then, thanks to (|5.5|) and the lower semicontinuity ol Nu 2 and the above convergence, 
we have, for e small enough, 

U2(t e ,Vs) < Nu 2 (t £ ,y E ). 

Therefore, at these points (t e ,y £ ), u 2 is a viscosity supersolution of the HJB equation 
given by 

— Vt + sup{— K (t, x, u) — f(t, x, u) ■ Dv} = , 

u£U 
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while v = fiui is a viscosity subsolution of the (slightly different) HJB equation 

—Vt + sup{— fJ,K(t, x, u) — f(t, x, u) ■ Dv} = . 

ueu 

By the results of the User's guide [8], there exists a € M such that 

(a,p £ ) £ D~u 2 (t e ,Vs) 
{a + $ t (t £> x £ ),p £ + D${t £ ,x £ )) e D + Ul {t £ ,x £ ) 

where p £ := ^ E ^ and therefore 

-a - $ t (t e ,x e ) + sup{- p,K(t £ ,x £l u) - f(t s ,x e ,u) ■ {D<S>(t £l x £ ) + p £ )} < , 
ueu 

-a+ sup{-K(t £ ,y £ ,u) - f(t £ ,y £ ,u) ■ p £ } > . 
ueu 

Subtracting these two viscosity inequalities, we get 

-® t (t e ,x e ) - sup{-K(t £ ,y £ ,u) - f(t e ,y £ ,u) ■ p £ } 
ueu 

+ sup{- pK(t £ ,x £ ,u) - f{t £ ,x £ ,u) ■ (D$(t e ,x e ) +p £ )} < 
ueu 

Using that sup(- • • ) — sup(- • • ) > inf(- •• — •••) and the continuity properties of K 
and /, we are lead to 

-$t(t s , x £ ) - F(\l + \x £ \)\D<j>(t £ ,x £ )\ - fiu K (\x e - y £ \) - u f (\x e - y £ \\p £ \) < 

Thus, as e — > 0, noticing that \x £ — y £ \\p £ \ = 2 ' Xe > 0, we obtain 

-$ t (i,x)-F(l + \x\)\D<l>(t,x)\ <0 

proving the claim that w = fj,u\ — U2 is a viscosity subsolution of (|5.4p in D T . 

Next we deduce that w < on [t,T] x D. We first remark that, by (A2), if 
x G D, \x\ < R + (3. Recall that K := F(l + R + /3). Hence io is as well a subsolution 
of 

(5.6) min{— Wt(t, x) — K\Dw\, w(t, x)} = in Z) T . 

We now construct a strict supersolution for this equation using "friendly giant" func- 
tions. For that, we consider an increasing C°° function ip : (— oo, (3) — > M such that 

ip(s) = if s < (3/2 and i/j(s) -> +oo as s -> /3. 

We claim that, for any k, xo € -D K , 77 > and To satisfying (|5.2p . the function 

Xx (i. x) := i>{K{T -t) + \x- x \) + r?(T - i) 

is a strict (smooth) supersolution of (|5.6p in the domain 

C K0 :={(t,z)e£> r : A-(T-t) + |a;-a;o| </?! # (T - i) < /3/2}. 

In fact this property can be checked by an immediate computation noticing that, for 
x = x , K(T — t) + \x — x \ = K(T — t)< (3/2 and therefore all the derivatives of ip 
are equal to at that point. Finally, for any k, set 

X {t, x) = inf Xxo {t, x) = ^{K{T -t) + d{x, D)) + V (T - t). 
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Then \ is well-defined and is still a strict supersolution of (I5.6[) in the domain 

C := {(t, x)eD T : K(T - t) + d(x, D) < 0; K(T -t)< f3/2}. 

Now we want to compare w and x m C. From the definition of x, it is clear that it 
tends to infinity on the boundary of the domain C. Using this remark together with 
standard comparison arguments yield that w < x m C. For any x G D, 

w(t, x) < X (t, x) = ip(K(T - *)) + r,(T -t) = r,{T - t) 

since K(T — t)< (3/2. Letting r\ tend to 0, we deduce that for x G D and r < t <T 

w(x,t) < 0. 

Hence m(r) < for all Tq < r < T. Thus m(To) < and the proof of the first step 
is complete. 

Proof of Step 2 : By the definition of M and N, and in particular the fact that g 
takes values in D, we deduce that, if x G A, then for any vq 

fiMui(t,x) < fxu!(t,g(x, v ))+fxC a (x K ,v ) < u 2 (t, g(x,v )) + C a (x K ,v ) + (fx- 1)C' , 

and therefore, taking the infimum on vq, fiMui(t, x) < Mu2{t,x) + (/x — 1)C". A 
similar property holds for Nui and Nu2- For the boundary points of A and C, we 
have to argue on the same lines as in Lemma 4.2, to show that the correct inequalities 
hold. 

We immediately deduce that w(t,x) < for T < t < T and x £ A because 
fiui(t, x) — u 2 (t, x) < fiMui(t, x) — Mu2{t, x) < 0. 

Proof of Step 3 : It remains to show that w < in [To, T] x (il \ A). To do so, we 
first prove that w is a subsolution of (|5.4p in the set {w > 0}. We only sketch this 
proof since it follows arguments we already used above. 

The only difficulty comes from the points (i, a;) in C since we may have the in- 
equality u 2 {x, t) > Nu 2 {x, t) (and not the inequality associated to the HJB equation). 
But, if this is the case, then, by Step 2, 

fiui(t,x) - u 2 (t,x) < fiNui(t,x) - Nu 2 (t,x) < (/x- 1)C < . 

Therefore this case cannot happen on the set {w > 0} and the same arguments as in 
Step 1 above show that w is a subsolution of (|5.4p in the set {w > 0}. 

To conclude, we have to modify the comparison argument by building "friendly 
giant functions" as the function x above but in a slightly different way in order to 
take into account the |x|-dependence in a suitable way. This also allows us to use a 
"local" comparison argument. 

For any interior point xo € £l K \ A K and for r, c > 0, we introduce the set 

O(x ,r) = {(t,x) e [T ,T] x ft : \x - x Q \ <r^c{t-T )} . 
Next we define x '■ 0(xq, r) — > M by 
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We need to show that x is a strict supersolution of (|5.4[) in O(x 0l r). By simple 
calculations, we get 



Xt 
D X 



(r a c(* - 2b) -|ar- &o| 2 ) 2 
2(z - x ) 

(r*c(t-T )-\x-x \ 2 ) 2 



Since |x| < \x — xq\ + \xq\ and \x — acol — r V C (T T))j we have 



2F(1 + M)|x - x | < 2F(1 + r % /c(T - T ) + \x \)r^/c(T - T ) 



Finally if we choose r > r(xo) = (1 + |xo|) and c such that *fc > i_2F(T-Ta) 1 we nave 

2F(l + \x\)\x-x \ <cr 2 . 

Note that such a choice of c is possible because of f !5.2j) . Substituting and using above 
inequality we can conclude that x 1S a strict supersolution of (|5.4ll in O(xo,r). 

By definition, x goes to oo on the lateral boundary of O(xo, r) and hence, on that 
part of the lateral boundary contained inside [To, T] x (Cl K \ A K ), we have no problem 
to compare w and x- Of course, w < x on [To, T] x A K since x > and u> < there 
by step 2. Therefore we can conclude that w < x m C(^o, r). 

Now we let r go to infinity to obtain 

w<0 in [Tj, T] x fl K \ A K , 

for every k. The proof is now complete since we can iterate the argument in time 
according to the fact that T — Tq depends only on f3 and K. □ 



Finally we show that the value function naturally satisfies the condition (15. ip 
under some reasonable additional assumptions. For that it is enough to show that 

\V(t,x) - h{x)\ -► 

as t approaches T, uniformly for bounded x G O. 

If t is close to T and |x| < r, it is easy to see that the trajectory X starting from 
x at time t, remains in a bounded subset of O on the time interval [t,T]. This is 
true even after the jumps, since the jump destination set D is bounded. Because of 
(C2), the number of jumps is finite and estimated uniformly in a; by a fixed number 
depending on r. Therefore, the integral of K gives a 0(T — t) whatever the trajectory 
is. 

Next we estimate V(t,x) — h(x) from below by considering a e-optimal control 
(u(-), v, £i, X(£i) ) (we drop the dependence with respect to e for the sake of simplicity 
of notations); we have 

T 

V(t,x)+s> I K(t,X(t),u(t))dt+ C a{X{rri),v) 

{ s<ai<T 

+ J2 C c (X(^),X(^y) + h(X x (T)) 

s<5,<T 
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But, using that 

h(X(s)) = Mh(X(s)) if X(s) G A 

or 

h(X(s)) = mm(h(X(s),Nh(X(s))) if X(s) G C, 

we deduce 

c c (x(er),^(6)') > ^(^(er)) - • 

Therefore 

F(t ;a; )+ £ >0(T-t)+ (kX(crr))-h(X(a+))) 

s<Ui<T 

+ Y (h(X(C))-h(X^)'))+h(X x (T)) 

s<£«<T 

In order to conclude, it is enough to rearrange the terms of the right-hand side and 
to make it appear as a sum of differences of values h between two jumps (after one 
jump and before the next one). The uniform continuity of h and the estimates on the 
/-ode (|2.1[) . lead to the desired property, since e is arbitrary. 
Conversely, for any w(-), we write 

T 

V(t,x)< f K(t,X(t),u(t))dt+ C a (X(ar),v) 

{ s<tn<T 

+ Y, C c (X(^lX(^)') + h(X x (T)) 

s<Zi<T 

but this time, when we touch either A or C, we choose the jumps in order to have 
C a (X(ar),v) < h{X{a7)) - h(X(a+))+s, 

c c (x(tn,x($i)') < Mmr)) - 

By the same arguments as above, we again obtain 

\V{t,x) - h(x)\ < m(T- t) for any |x| < r 

where m(r) — > as r — * depend only on the L°° norm of K and on the modulus of 
continuity of h on the bounded subset where the trajectory X lives. 

The above inequalities show that V assumes the terminal data h uniformly w.r.t 
i, which is exactly (|5.ip . 
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